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ABSTRACT 

Panoramic  cameras  offer  Hue  real-time.  360-degree 
coverage  of  the  surrounding  area,  valuable  for  a  variety  of 
defense  and  security  applications,  including  force 
protection,  asset  protection,  asset  control,  security 
me  luding  pent  security,  perimeter  security,  video 
surveillance,  border  control,  airport  security,  coastguard 
operations,  search  and  rescue,  intrusion  detection,  and 
many  others.  Automatic  detection,  location,  and  tracking 
of  targets  outside  protected  area  ensures  maximum 
protection  and  at  tlx-  same  time  reduces  the  workload  on 
personnel,  increases  reliability  and  confidence  of  target 
detection,  and  enables  Kith  man-in-thc-loop  and  fully 
automated  system  operation.  Thermal  imaging  provides 
the  benefits  of  all-weather,  24-hour  day  night  operation 
with  no  downtime.  In  addition,  thermal  signatures  of 
different  target  types  facilitate  belter  classification, 
beyond  the  limits  set  by  camera’s  spatial  resolution. 

The  useful  range  of  catadioptric  panoramic  cameras 
is  affected  by  theu  limited  resolution.  In  many  existing 
systems  the  resolution  is  optics-limited.  Reflectors 
customarily  used  in  catadioptric  imagers  introduce 
abenations  that  may  become  significant  at  large  camera 
apertures,  such  as  required  in  low-light  and  thermal 
imaging.  Adv  antages  of  panoramic  imagers  with  high 
image  resolution  include  increased  area  coverage  with 
fewer  cameras  instantaneous  full  horizon  detection, 
location  and  tracking  of  multiple  targets  simultaneously, 
extended  range,  and  others. 

The  Automatic  Panoramic  Thermal  Integrated  Sensor 
(API  IS),  being  jointly  developed  by  Applied  Sckikc 
Innovative,  Inc.  (ASI)  and  the  Armament  Research. 
Development  and  Engineering  Center  (ARDEC) 
combines  the  strengths  of  improved,  high-resolution 
panoramic  optics  with  thermal  imaging  in  the  X  14 
micron  spectral  range,  leveraged  by  intelligent  video 
processing  for  automated  detection,  location,  and  tracking 
of  moving  targets.  The  wotk  m  progress  supports  the 
Future  Combat  Systems  (ECS)  and  the  Intelligent 
Munitions  Systems  (IMS). 

lire  innovative  automated  thermal  imager  with  360" 
field  of  view  (FOV)  optics  provides  situational  awareness 


and  instantaneous  full  hon/on  detection,  location,  and 
tracking  of  multiple  targets.  It  features  pixel  limited 
resolution,  advanced  sensor  node  in  a  network  of 
intelligent,  distributed  sensor  arrays.  Integration  of 
commercial  off  the  shelf  (COTS)  components  is 
maximized  in  the  APTIS  design  to  reduce  Cost  and  tnix  to 
market.  The  APTIS  is  anticipated  to  operate  as  an 
intelligent  node  in  a  w  ireless  network  of  multifunctional 
nodes  that  work  together  to  serve  in  a  wide  range  of 
applications  of  homeland  security,  as  well  as  serve  the 
Army  in  tasks  of  improved  situational  awareness  (SA1  ui 
defense  and  offensive  operations,  and  as  a  sensor  node  m 
tactical  Intelligence  Surveillance  Reconnaissance  (ISR). 

The  novel  ViperView'”  high-re  solution  panoramic 
thcmial  imager  is  the  heart  of  the  APTIS  system.  It 
features  an  aberration-corrected  omnidirectional  imager 
with  small  optics  designed  to  natch  the  resolution  of  a 
640x480  pixels  IR  camera  with  improved  image  quality 
for  longer  range  target  detection,  classification,  and 
tracking.  The  same  approach  is  applicable  to  panoramic 
cameras  working  m  the  visible  spectral  range.  Other 
components  of  the  ATPIS  system  include  network 
communications,  advanced  power  management,  and 
wakeup  capability.  Recent  developments  include  image 
processing.  Optical  design  being  expanded  into  the  visible 
spectral  range,  and  w  ireless  communications  design.  This 
paper  describes  tlx*  development  status  of  the  APTIS 
system. 

I.  INTRODUCTION 

Applied  Science  Innovations.  Inc.  (ASI)  and  the 
Armament  Research.  Development  and  Engineering 
Center  (ARDEC)  are  jointly  developing  tlx-  Automated 
Panoramic  Tlx-imal  Imaging  Sensor  (APTIS)  •  a  new- 
thermal  onager  with  360°  field  of  view  small  optics  to 
provide  instantaneous  full  horizon  detection,  location  and 
tracking  of  multiple  targets1,2.  Ihe  innovative 
ViperView11  optical  design  of  tlx-  onager  provides  for 
small  size  and  low  cost,  combined  with  tlx-  high 
resolution,  matching  pixel  resolution  of  Ihe  state-of-the- 
art  utKoulcd  thermal  infrared  sensors.  Tlx-  high 
resolution  is  key  ui  increasing  the  target  detection  range, 
extended  target  identification  range,  and  reduced  target 
acquisition  lime.  Tlx-  integrated  APTIS  imager  design 
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includes  global  positioning  system  «iPS|  navigation  and 
Communication  interfaces  to  serve  as  an  advanced  sensor 
node  in  a  network  of  intelligent,  distributed  sensor  arrays. 
Power-save  mode  and  remote  vvakcup  capability  will 
serve  lor  extended  unattended  operation. 

Conventional  cameras  luve  limited  Held  of  view  in 
the  horizontal  dimension,  failing  to  display  objects  and 
ev  ent  beyond  the  FQV.  Unlike  this  "tunnel  vision-  of  a 
camera  with  conventional  optics,  human  vision  includes 
peripheral  vision,  with  the  filed  much  wider  than  the  high* 
resolution  portion  of  the  field  around  the  forward 
direction.  Peripheral  vision  allows  us  to  readily  detect 
large  objects,  contrast  colon,  and  fast  motion.  Pan-tilt- 
zoom  (PTZ)  cameras  may  cover  full  hon/on.  but  fall 
short  of  full-time,  full  horizon  coverage  and  are  prone  to 
nussuig  rapid  events. 

Ihe  ViperView™  thermal  imager  of  tire  A  IM  IS 
sensors  may  serve  to  complement  a  separate  conventional 
camera  to  mimic  human  vision  and  exceed  its  capabilities, 
with  full-time  360°  coverage,  day.'night.  all-weatlrer 
Operation,  and  automatic  target  detection  and  tracking  for 
maximal  situational  awareness. 

High  image  resolution  of  a  panoramic  imager  in  a 
sensor  ensemble  increases  the  coverage  area,  extends  the 
range  of  effective  target  detection,  location,  aiul  tracking, 
reduces  tire  number  of  senior  nodes  necessary,  thus 
reducing  the  overall  east.  Existing  panoramic  imagers, 
e  g.,  with  parabolic  or  hyperbolic  reflectors,  have  optical 
aberrations,  which  typically  arc  reduced  by  imaging  at 
low  image  numerical  apertures.  This  approach  is  well 
known  in  optics  and  photography:  at  small  apertures,  any 
Ions  operates  in  a  "near-pinhole”  mode,  this  approach  to 
reduction  of  aberrations  comes  with  a  significant  loss  of 
light.  Reflector  aberrations  may  significantly 
Compromise  image  resolution,  especially  at  high 
apertures.  High  apertures  ("fast"  optics!  are  required  in 
such  applications  as  low  light  imaging  and  tlrermal 
imaging.  In  low  light,  large  optical  apertures  translate 
into  higher  light  throughput.  In  thermal  imaging,  the 
diffraction  spot  size  increases  as  lire  ratio  of  the 
wavelength  to  tire  aperture  diameter.  flic  longer 
wavelength  of  the  infrared  range  therefore  requires  larger 
apertures  to  offset  tire  effect  of  tire  longer  wavelength  and 
keep  the  optical  resolution  in  a  match  with  Ihe  pixel 
resolution  of  the  sen  or.  lire  largo  aperture  of  the  optics 
reduces  the  effect  of  diffraction  and  improves  image 
quality.  In  panoramic  catadioptrre  systems,  the  minor 
size  affects  tire  effective  aperture.  High  image  quality  is 
easy  to  achieve  with  large  mirrors  placed  at  long  distances 
from  the  camera.  In  other  words,  largo  eatadioptnc 
panoramic  cameras  have  lower  aberrations  and  produce 
better  images,  at  the  expense  of  the  overall  larger  size  of 
the  optics.  On  the  other  hand,  small  panoramic  optics 
with  low  aberrations  presents  a  design  challenge,  l  ire 


challenge  is  even  higher  for  thermal  optics,  where 
diffraction  of  the  longer  wavelength  dictates  large 
apertures  regardless  of  Ihe  light  throughput,  in  order  to 
match  the  diffraction  limited  spot  size  with  Ihe  pixel  pitch 
of  the  sensor. 

Unless  aberrations  of  Ihe  optics  are  corrected,  the 
image  spot  size  may  significantly  exceed  the  pixel  size  of 
tire  scinur.  compromising  tire  investment  ui  a  high- 
resolution  long -wavelength  infrared  (LWIKl  sensor.  In 
tire  A  IMIS  panoramic  thermal  imager,  effective  collection 
of  thermal  IR  radiation  necessitates  high  nunrerical 
apertures,  exceeding  those  typical  of  existing  panoramic 
imagers.  Ihe  requirement  for  small  size  of  Ihe  sensor 
calls  for  reduced  mirror  size.  Our  development  included 
optical  design  effort  that  resulted  in  an  integrated 
panoramic  imager  with  small  optics,  large  effective 
aperture  for  maximum  collection  of  IR  radiation,  and 
uneomprorniNcd  image  resolution,  matching  the  resolution 
of  a  MO  x  480  LWIR  sensor. 

2.  DESIGN  APPROACH 

AP  I  IS  is  a  new  automated  theimal  imager  with  360 
field  of  view  (FOV)  optics  to  provide  instantaneous  full 
horizon  detection,  location  and  tracking  of  multiple 
targets.  The  small,  high-resolution  optics  of  the  sensor 
has  been  designed  to  match  the  focal  plane  airay  thermal 
imager  with  640x480  pixels  potential  to  provide  day  night 
detection  of  personnel,  aircraft  and  vehicles  even  when 
camouflaged.  Ihe  high  resolution,  low  cost  imager  will 
provide  for  longer  target  detection  range,  extended  target 
identification  range,  and  reduced  target  acquisition  times. 
The  novel  combination  of  llur  high-resolution  thermal 
imager  with  361/’  FOV  optics  will  enable  accurate  target 
hearing,  temperature  profiles,  and  rough  order  of 
magnitude  target  imaging  which  can  aid  classification, 
discrimination  and  identification  of  targets. 

The  new  panoramic  optics  of  tire  theimal  imager  has 
been  designed,  built,  and  demonstrated.  Development  of 
tire  integrated  sensor  is  now  under  way  through 
integration  of  Ihe  panoramic  imager  with  other 
components  including  GPS.  digital  compass,  till  sensor, 
target  image  processing,  and  wireless  communications  for 
ad-hoc  networking.  All  of  these  components  arc  available 
commercially. 

The  approach  selected  m  this  development  is  based 
on  the  VipcrV'rew  panoramic  thermal  imager  with  3MI- 
doegree  horizontal  field  of  view  and  small,  high- 
resolution  optics  foe  maximum  detection  range.  Ihe 
APT1S  design  is  optimized  for  operation  ui  a  "man  m  the 
loop"  network  of  intelligent  sensor  nodes.  Automated 
detection  of  targets  in  protected  area  reduces  w  orkload  on 
lire  human  operator. 


Pit*  integrated  APTIS  sensor  suite  it  designed  fur 
field  deployment  with  battery-powered  imager  senior 
nodes.  Advanced  power  management  subsystem  is 
designed  to  contcrvc  the  power  provided  by  the  buttery. 
Components  of  tlie  API  IS  lysieiu  lull  into  three  major 
groups,  based  on  the  level  of  buttery  power  consumption. 
The  lint  gtoup  includes  wakcup  and  power  munugemenl. 
consummg  negligible  amount  of  battery  power,  such  that 
the  buttery  life  with  these  sensors  alone  is  practically  the 
same  as  the  shelf  life  of  the  battery.  A  portion  of  network 
Communications  (in  lb.  receiver  mode)  is  included  in  this 
gioup  as  well  tor  getting  cues  form  other  sensor  nodes. 
This  first  group  of  components  n  "Always  On-  to  provide 
a  wakcup  signal  to  the  thermal  imager  and  other  "power 
hungry"  components. 

fhe  second  group  includes  a  Global  Positioning 
System  KiPSt  receiver,  which  operates  intermittently. 
e.g.  once  every  10-15  minutes.  The  reason  lor  this 
operation  mode  is  that  it  lakes  a  GPS  receiver 
approximately  one  minute  on  average  to  acquire  satellites 
before  the  latitude  and  longitude  can  be  determined  and 
store  in  the  memory  of  the  APTIS  processing  hardware. 
An  additional  hincUon  of  the  GPS  component  is  liming 
synchront/utinn  over  the  network. 


tracking,  and  the  wireless  network  eonunumcations.  The 
electronic  compass  and  the  uli  sensor  are  also  included  ui 
this  group.  Ihis  group  is  powered  once  at  the  time  of 
deployment.  In  combination  with  tlic  GPS.  electronic 
compass,  and  the  tilt  senior,  this  brief  initial  operation 
results  in  a  detemunabon  of  the  location  and  onentalion 
of  the  unit  in  space,  as  well  as  provides  learning  of  the 
surrounding  area  by  the  intelligent  sensor  software 
responsible  for  automatic  detection,  location,  and  tracking 
of  targets.  Hi  is  learning  process  Serves  for  effective 
target  discrimination  agamst  background,  including 
moving  objects  such  as  shrubs  and  trees.  The  electronic 
compass  and  the  tilt  sensor  provide  to  the  processor  unit 
the  true  North  direction  and  the  vertical  axis,  necessary 
tin  translating  pixel  coordinates  of  the  target  into  the 
absolute  location  and  direction  to  target.  The  thud  group 
of  sensor  component  also  powers  up  on  every  wakcup 
signal  (nun  the  first  group. 

Once  the  wakcup  signal  brings  the  integrated  APTIS 
sensor  to  full  operation  mode,  the  panoramic  thermal 
imager  acquires  images  at  the  video  frame  rate  of  30  (ps 
ami  sends  the  video  Stream  10  the  processor  unit.  Tlie 
processor  detect*  moving  targets  from  the  v  idco  siream. 
The  software  is  capable  of  intelligently  separating 
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meaningful  target  nweion  fkim  motion  clutter,  such  a% 
produced  by  Irect.  thrub*.  and  similar  natural  objects. 
The  software  use*  a  set  of  rules  for  detection  ami 
identification  of  targets.  The  rules  can  be  fine  tuned  by 


the  operator  at  any  time.  They  include  criteria  such  as 
target  location,  speed,  direction  heading,  sr/e.  altitude, 
and  confidence. 


A  remote  terminal  receives  over  a  wireless  link  real¬ 
time  imagery  acquired  by  the  panoramic  camera.  The 
terminal  is  equipped  w  ith  software  for  image  processing, 
which  includes  image  translbciiuliun  ("unwrapping-), 
with  targets  highlighted.  In  addition,  up  to  five  regions  of 
interest  unrounding  targets  are  shown  separately. 

For  every  detected  target,  communicated  to  tlx: 
central  node  (remote  terminal,  man-m-the-loop)  are  I) 
target  coordinates,  speed,  direction  headmg.  and  altitude; 
2)  image  regions  of  interest  (ROI)  containing  the  targets 
or  target  clusters;  3)  "unwrapped"  full  frames,  with 
targets  marked,  in  near-real  time  as  provided  by  the 
bandwidth  of  the  communications  channel.  This  allows 
the  operator  to  identify  and  track  targets,  as  well  as  direct 
other  senior*  and  assets  to  the  target  or  targets  as 
necessary 

Mechanical  design  of  the  APTIS  prototype  includes  a 
lightweight,  low  cost  mounting  adapter  that  eiubles 
several  iixiunting  options  of  the  imager.  The  mam 
mounting  options  are  "ground  low"  on  a  small  tripod, 
providing  the  "guard  dog's  point  of  view"  compatible 
with  inunilioiis  systems  and  unattended  ground  sensor 
applications;  "ground  high"  on  a  tiipod  6  to  X  feet  high,  in 
a  reverse  orientation  lor  area  monitoring  form  an  elevated 
point;  and  vehicle  mount  of  application  on  regular  and 
unmanned  ground  vehicles,  applicable  to  remotely 
controlled  robots  and  to  v  ehicle  protection. 

Alter  a  preset  period  of  tune  with  no  targets  beuig 
delected  by  the  imager,  the  imager  and  the  other 
components  of  the  "power  hungry"  group  ate  turned  ulY 
until  another  wakcup  signal. 

3.  PANORAMIC  IMAGE  UNWRAPPING 

Pictures  taken  with  the  panoramic  camera  contain 
360'  surrounding,  views,  flic  "donut"  image  is  not  easy  to 
view  for  the  human  operator.  To  improve  the  visual 
viewing  effect,  image  unwrapping  is  needed  to  present  tlx: 
obtained  panoramic  camera  into  a  360"  unwrapped 
panoramic  view.  In  the  Al’flS  application,  near  real  time 
imagery  is  to  be  unwrapped  with  virtually  no  additional 
lime  delay.  Fin  this  purpose,  fast  image  processing 
software  routine  has  been  developed  for  image 
unwrapping.  Figure  2  shows  the  approach  used  ui  the 
unwrapping  algorithm. 
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Figure  2.  Principle  of  the*  panoramic  linage  unwrapping 

The  "donut"  image  is  mapped  onto  the  output  frame 
using  coordinate  transformation  from  polar  to  rectangular. 
The  fast  mapping  routine  was  written  in  C  and 
implemented  on  a  PC  computer.  Transformation  of  a 
640x480  24-bit  linage  was  demonstrated  at  franc  rales  in 
excess  of  20. 

Figure  3  shows  an  uiugc  taken  with  a  panoramic 
camera.  Ihe  image  size  is  640  x  4X0  pixels.  Unwrapping 
this  image  into  a  rectangular  panoramic  view  results  in  a 
reproduced  panoramic  picture  with  a  maximum  height  of 
240  pixels,  equal  to  Ihe  "donut"  serm-diameter  of  Ihe 
picture  ui  Figure  3.  lire  w  idlh  of  Ihe  unwrapped  image  is 


Figure  3.  Photo  taken  by  the  panoramic  imaging  sensor 
with  visible  light. 


Figure  4  shows  the  unwrapped  image  of  Figure  3.  It 
is  Ihe  lull  360-degree  view  of  the  scene.  The  sr/e  of  Ihe 
picture  is  15144  x  240.  with  a  minor  overlapping  at  the 

ends. 
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Figure  4.  Unwrapped  image,  image  sue  is  15K4  x  240 

4.  DETECTION  RANGE 

Ihe  detection  range  is  a  function  of  the  number  of 
pixels  on  the  target  and  the  angular  magnification  of  tlx: 
optics.  The  number  of  pixels  required  varies  with  the 
target  to  background  contrast  and  depends  on  the  unage 
quality  produced  by  the  camera,  which  also  atlccls  the 
contrast  (modulation  transfer  function,  or  MTF).  Other 
factors  affecting  the  detection  range  include  the  detection 
algorithm,  which  distinguishes  the  "meaningful-  motion 
of  Ihe  target  from  background  motion,  such  as  trees  or 
shrubs  in  Ihe  wind,  living  bmb.  and  sunilar  natural 
objects  and  events. 


In  addition.  Ihe  number  of  pixels  required  anil  l lie 
related  detection  range  are  a  result  of  a  tradcolV  that 
involves  the  rate  of  false  alannv  Considering  all  of  these 
factors,  it  is  reasonable  to  assume  two  pixels  across  the 
horizontal  dimension  of  the  target  as  an  estimate  of  iIk 
required  number  of  pixels  tin  automatic  ikMedhsn. 


figure  5.  Effect  of  the  FOV  and  hon/on  location  on 
resolution 


With  this  assumption,  the  detection  range  can  be 
estimated  as  follows. 
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Here  La,  is  the  detection  distance;  d  target  size; 
n„,  number  of  horizontal  pixels  on  target  required  by 
the  software  for  detection;  R  -  radius,  in  pixels,  of  the 
image  of  horizon  on  the  sensor.  Ihe  latter  radius  can  be 
estimated  as 
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where  R„,  and  R„„  are  the  "donut”  radii,  as 
illustrated  by  figure  5;  VFOV,  and  VFOV.  -  positive 
Cup”)  and  negative  (“down”)  vertical  fields  of  view  of 
the  imager. 

Assuming  an  upright  person  being  the  target 
(shoulder  width  d-SQcm;  2  pixels  required  for  detection). 
Ihe  following  values  can  be  used  for  estimating  the 
detection  range:  d  -  50cm;  n^,  «,  -  2;  VFOV'i  -  45”: 
VFOV;  -  -5";  iy  “  4W>  -  120.  R~.  - 

Using  these  values  in  fonnulas  (11  (2).  one  arrives  at  the 
detection  range  of  approximately  57  meters  It  should  be 
noted  that  a  compromise  exists  between  the  “down" 
vertical  field  of  view  VFOV,  and  the  detection  range. 
When  VFOV,  -0.  Lx,  is  60  meteis:  if  VFOV.  -  -15deg. 
Lfl,  is  52  meters,  if  VFOV,  -  -30deg.  lJo  is  40  meters, 
and  so  on.  This  tradeoff  is  somewhat  counter-intuitive 
and  should  be  taken  into  account  when  choosing  the 
mounting  method  for  the  onager.  If  the  camera  is 
installed  at  an  elevated  point,  such  as  a  tripod,  one  might 
intuitively  expect  a  longer  detection  range.  While  this  is 
Hue  in  terms  of  avoiding  obstacles  and  obscurations 
covering  the  target,  the  minimal  field  of  view  needs  to  be 
chosen  below  zero  to  cover  low  elevation  points  near  the 
camera.  I  he  associated  reduction  in  value  R  (image  of 
the  ho ii/on  on  the  sensor;  Figure  51  reduces  the  detection 
range  of  targets  approaching  from  the  horizon  at  or  near 
zero  elevation  angles. 
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Figure  6.  ViperView  optics  •  experimental  demonstration:  a)  experimental  optical  system;  b)  sample  image  produced 
by  ihe  camera;  c)  zoomed -in  I  ragmen  I  of  the  same  image  showing  pixel- luniicd  resolution 


5.  ABERRATION-CORRECTED  VIPERVIEW 
OPTICS 

A  mayor  problem  m  existing  caladioptnc  panoramie 
optical  tyklOlU.  especially  working  at  large  apcilures.  is 
die  resolution  limitation  imposed  by  optical  components 
flenses  and  minors).  W  itli  cities- limited  resolution,  suth 
panoramic  cameras  cannot  benefit  from  high  resolution 
imaging  sensors  |M0x480  pixels  or  higher).  Even  at 
lower  resolutions.  some  of  the  existmg  panoramic 
cameras  have  optics-limited  resolution  ViperView 
optics,  along  with  the  small  size,  features  aberration 
correction  to  a  level  adequate  to  match  a  640x480 
imaging  sensor. 

Optical  design  of  the  ViperView  imager  was 
performed  by  Applied  Science  Innovations  using  the 
OSLO  Premium  software.  Hie  orientation  of  the  "donut" 
unage  on  Ihe  sensor  was  selected  such  that  the  positive 
I  above  horizon)  elevations  projected  towards  the  center, 
whereas  negative  (below  hurizun)  elevations  pruyected 
towards  tlie  penpheiy,  witli  the  horizon  imaged  close  to 
ihe  periphery  of  the  image  as  well.  Ibis  image 
onentalion  provided  for  optimal  allocation  of  senor 
resoluUim.  with  the  larger  number  of  pixels  for  imaging 
Ihe  hori/on.  where  targets  are  more  likely  to  be  located. 
Tlie  altemativ  e  image  orientation,  with  Ihe  direction  up 
imaged  lovvanls  the  outside  of  the  "donut",  w  as  rejected 
as  sub-optimal,  with  lower  pixel  count  imagmg  tlie 
hori/nn.  luwcr  maximal  elevation,  ami  significant  portion 
of  ihe  pixels  imaging  ihe  ground  in  a  close  v  icirniy  of  the 
sensor.  Optimizalnm  of  Ihe  unage  quality  was  perlormed 
while  imintainmg  small  size  of  tlie  optics  and  the  desired 
vertical  field  of  view.  Several  optical  designs  were 
produced,  with  the  Spot  size  smaller  than  the  pixel  pitch 
uf  a  640x480  sensor  over  the  complete  field 

Experimental  miplementation  of  the  ViperView 
optics  and  wimple  images  picduccd  by  it  are  shown  in 
F  igure  6. 


The  thermal  camera  used  in  these  preliminary 
experiments  had  the  pixel  resolution  320x24(1 
Integration  of  a  high-resolution,  640x480  uncooled 
theimul  imager  mto  tlie  APTIS  prototype  is  now  under 
way.  The  large  numerical  aperture  of  tlie  optics  was 
equivalent  to  that  of  a  regular  1.0.85  lens  As  illustrated 
by  ihe  images  captured.  Ihe  resolution  is  clearly  pixel 
limited,  which  proves  that  alienations  of  tlie  optics  are 
well  corrected. 

6.  AUTOMATIC  TARGET  DETECTION  AND 
TRACKING 

The  v  idco  stream  from  the  panoramic  camera  is  sent 
to  tin1  video  processor  board  for  detection,  location,  and 
tracking  of  targets.  Detection  is  preformed  on  the  camera 
end  of  the  communication  link,  on  Ihe  "unwrapped", 
"donut”  unage.  The  processor  Ixiard  uses  field- 
programmable  gale  arrays  (FPGA)  wtlh  intelligent  video 
processing  unplemcnled.  Sev  eral  processing  modes  and 
detection  and  tracking  tules  are  available  The  choice  of 
the  processing  algorithm  anil  lute-tuning  of  tlie  detection 
rules  are  preformed  from  Ihe  camera  host  computer  over 
ihe  standard  serial  interface.  Currently,  tlie  rules  are  set 
poor  lo  operating  the  APTIS  system,  using  the  camera 
computer.  Future  development  will  include  providing  the 
rale  adjustment  remotely,  over  the  wireless 
communications  Imk. 

The  intelligent  algorithm  running  on  the  FPGA  board 
automatically  detects  and  trucks  up  to  live  targets.  Tlie 
input  video  is  analog,  in  tin*  NTSC  formal  The  FPGA 
has  a  processing  ability  of  horizontal  resolution  of  720 
pixels  and  tlie  vertical  resolution  of  242  pixels  per  field, 
i.e..  484  lines  per  frame,  at  60Hz  as  provided  by  tlie 


NTSC  standard.  The  image  includes  a  rectangular  region 
containing  Ihe  361)  ikrgree  panoramic  view  of  tlx: 
surrounding  environment.  Tlx*  FPGA  hoard  outputs  an 
analog  video  signal  overbid  with  detection  symbology. 
The  symbology  and  the  target  information  are  also  served 
Ihtxiugh  serial  data  interface  for  target  screening  and 
further  processing  purposes. 

Target  acquisition  aixl  tracking  can  be  performed  in 
different  detection  and  tracking  mods*.  A  primary  target 
can  be  iifeiililicd  according  to  different  criteria  while 
information  on  additional  targets  is  also  served 
continuously,  providing  multiple  target  tracking.  Multiple 
criteria  for  selecting  Ihe  primary  target  are  available, 
including  location,  intensity,  speed,  and  others.  In  the 
thermal  imagery  produced  by  the  VlperView  camera,  the 
latget  intensity  and  color  (shade  of  grey)  characterize  the 
thermal  signature  of  Ihe  target.  This  important 
characteristic  is  used  for  target  detection  and  tracking 
along  with  other  selection  criteria.  The  latget  detection  is 
fast,  in  the  fractions  of  a  second  lime  frame,  for  both  new 
targets  aixl  targets  reappearing  after  hiding  or  being 
obstructed. 


encrypted  foe  secure  transmission  preventing 
unaUhon/cd  access.  Components  are  housed  in 
weatherproof  cases.  Ihe  system  consumes  3W  power  at 
each  power  injector  when  communicating.  The  output 
power  reaches  400  mW. 

All  of  the  hardware  on  the  camera  side,  including 
computer,  camera.  FPGA  board,  aixl  wireless  transmitter 
is  pul  into  hibernation  at  the  client  operator's  command, 
provided  no  targets  of  interest  are  in  range.  To  reduce  the 
power  consumption  as  much  as  possible,  tlx  wireless 
networking  will  cease  to  function  when  the  AIT  IS  server 
is  in  hibernation,  in  which  power  supply  to  the  server 
computer,  the  wireless  network  hardware,  the  til’s 
receiver,  and  the  electronic  compass  )  tilt  sensor  will  be 
disconnected.  Wakeup  of  the  AITIS  server  is  provided 
by  an  ultra  low  power  wireless  wake  up  device  currently 
being  integrated  into  the  prototype.  At  Ihe  operator's 
command  on  the  client  laptop,  tlx  receiver  wakeup  device 
on  the  server  side  will  Switch  on  Ihe  ATX  power  supply  at 
the  server,  to  resume  the  power  supply  to  all  Ihe  API  IS 
components.  The  embedded  computer  will  resume  all  of 
Ihe  functions  it  was  performing  when  put  into  hibernation. 


Figure  7.  "Unwrapped "  thermal  imagery  is  displayed  in  the  A I  PIS  GUI.  w  ith  moving  targets  marked  and  target  data 
registered  in  real  lime. 


8.  WIRELESS  COMMUNICATIONS 

The  APT1S  panoramic  camera  is  hinted  by  an 
embedded  PC  1 04  «  computer  running  a  server  application 
to  reby  collected  data  to  a  client  computer. 
Communications  between  this  server  ami  the  operator 
client  lugged  notebook  computer  arc  established  through 
a  secure  wireless  network.  The  w  ireless  hlhemct  is  based 
on  standard  K02.1la  technology  and  Operates  in  the  5.3 
and  5.8  (ill/  U-NII  hands.  The  system  of  dual  RF 
modules  is  as  a  plug  and  play  CATS  Ethernet  cable 
replacement  designed  for  use  with  Standard  Ethernet 
devices.  The  system  transmits  from  2500  feet  to  30  miles 
at  20  Mbps  digital  data  stream  in  standard  mode  and  a  40 
Mbps  digital  data  stream  in  turbo  mode.  With  a 
moderate- si/e  antenna,  the  system  operates  up  to  4  miles 
m  light-of- sight  conditions.  The  communication  is 
currently  determined  by  the  MAC  address  of  each 
transceiver.  Ihe  signal  between  Ihe  RF  nxidules  is 


CONCLUSIONS 

Development  of  ihe  Automatic  Panoramic  Thermal 
Imaging  Sensor  is  under  way.  The  sensor  is  designed  as 
an  intelligent  node  in  an  ad-hoc  sensor  network.  The 
design  features  automatic  detection,  location,  aixl  tracking 
of  multiple  targets  and  advanced  power  management  for 
extended  battery  life,  enabled  by  remote  wakeup 
capability.  ViperView’'1,  tlx:  small,  high-resoluliun 
imager  of  the  APTIS  system  has  been  designed, 
implemented,  aixl  demonstrated  experimentally. 
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